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Criticism of the conference model should be put in context. Evidence suggests that the 
essential features of this model have emerged as responses to challenges posed by current 
trends of scientihc research and the impact of the new techno-economic paradigm: the age of 
Information and Communication Technology |H]. The economic, social and technical impact 
of this revolution on the whole scientihc research cycle is beginning to be assessed mm- 
At the center of it is our discipline, computing science (CS), that in this process is rehecting 
on its identity and relationship with other disciplines mm- 

This context seems indispensable when discussing today’s problems of scientihc evalu¬ 
ation, in particular the Conference vs. Journal (CvJ) debate. This debate, also, would 
beneht from systematic historical and sociological studies of these practices. This ground 
would help to ponder the criticism of the conference model by showing that a great per¬ 
centage of the problems like the rapid increase in the number of submissions, skimpy and 
slow reviews, declining paper quality, pressures from academia and publishers, are ahecting 
the whole system, be it conference or journal mm- In this article we briehy develop these 
arguments. 

Evaluation in Classical Sciences. Although their origins can be traced to scientihc 
societies, it is in the nineteen century that the journal and peer review models as we know 
them today emerged. The system evolved without major changes until the second half of 
the 20th century, when it was shaken by the new printing and communication technologies 
(photocopies, email, Web, etc.). Editorial boards evolved from a group of people whose 
main goal was to hnd enough material to hll the pages of the journal, into a huge hltering 
system whose goal turned to enforce minimum standards and discard unsound works [2]. 
Recognized today as the essence of scientihc evaluation, peer review has been lately the 
center of heated discussions in all disciplines |1] |9l |12]. The journal model, in its origin had 
two driving forces: the need to incorporate more articles into the publishing process, and to 
encourage scientists to disclose their work (by giving credit and prestige to the author) [3]. 


1 


The first one is not needed anymore, bnt a new one has emerged: to measnre “prodnctivity” 
in a system of science where acconntability plays an increasing role m- 

Regarding onr CvJ debate, another facet of this history seems more relevant. Scientihc 
societies held periodic meetings where their members presented their works and advance¬ 
ments in their disciplines. At the beginning, journal contents were almost a compilation of 
these presentations. The method did not scale and these practices split: on one hand today’s 
journals, and on the other, periodic meetings of the society where members used to present 
their hndings and ideas. Journals got under evaluation methods to check the relevance of 
submissions, while meetings kept a natural hlter: to be member of the society. These two 
forms of communicating scientific research made their own way. 

The advent of Conferences. For some sociological reason, a strange event, a mix of 
meetings and journals, was developed in our discipline. Opinions and remembrances ascribe 
this phenomena to the particular nature of the “products” of the discipline; the perception 
that CS has a different speed from other sciences; the few journals available and the long 
delays in refereeing and publishing; facilities given by traveling in the jet age; the avoidance 
of wasting time in polishing for a journal material already presented in conferences. (See 
CACM, vol 52, Nos. 1,4,5,8 and links from there). The need to test these hypotheses 
becomes evident when reading the most widespread of them: “Quick development of the held 
required quick review and distribution of results”, which although intuitive, does not explain 
conferences {Letters in other helds give much “faster” review and dissemination times). The 
phase transitions of this process are also important. Table [H showing rough statistics based 
on the information of the ACM Digital Library, indicates that conferences/workshops are 
inhuential in the held at least since the early 1970’s when the numbers exploded. The 
series LNCS, which publishes mainly CS events proceedings, began in the seventies with a 
dozen a year and since then it has grown to reach currently more than hve hundred a year. 
The data, although illustrative, is by any means apt to derive dehnitive conclusions, because 
parameters like attendance, impact, spreading, relevance, etc. of these events is not weighed. 
Dehnitively a systematic study is needed to put the CvJ debate on solid grounds. 

Conferences vs. Journals. A big part of the discussions on CvJ focuses on criticism 
on the current conference model. Bad quality of papers, splintering of communities, few 
attendees, reviews done under extreme time and workload pressures, are among the typical 
complaints. Many of them are also found in current criticism of journals. Thus to contribute 
to the debate, it seems important to isolate characteristics that are inherent to conferences. 
Below we present a hrst approximation. 

• Time constraints for the researcher. Enforcing research deadlines puts time pressure. 
On the contrary, journals allow submissions at any moment, giving more freedom and 
relaxed pace to research. Deadlines are slightly being incorporated into journals, e.g. 
“special issues”, Proc. of the VLDB Endowment, etc. Regarding time constraints for 
reviewers, the conference practice is being increasingly adopted by journals, by fixing 
review times. 
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Period 

Jonrnals 

Conf. (still existing) 

Conf. (not existing today) 

Before 1960 

1 

0 

2 

1960-64 

0 

2 

0 

1965-69 

1 

7 

0 

1970-74 

0 

11 

3 

1975-79 

4 

4 

2 

1980-84 

3 

8 

2 

1985-89 

0 

23 

14 

1990-94 

4 

22 

18 

1995-99 

3 

46 

13 

2000-04 

9 

55 

25 

2005-09 

13 

38 

6 


Table 1: Number of ACM Journals and Conferences by birth date 


• Space constraints. Limiting the size of articles puts pressure on the form of commu¬ 
nication. Only the essential without details is presented. The motivations seem to 
be time review constraints and fast communication. This notion is rather foreign to 
classical journals, although Letters have incorporated it long ago. 

• Boolean decisions. This is one of the distinctive characteristics of the conference model. 
As the number of submissions grow, many journals are incorporating sharper decisions 
methods (limited resubmission rounds, etc.) Under the think/act dichotomy, this 
method incentives action rather than deliberation. 

• Discussion post-publication. Real testing of hypothesis begins after the conference 
(where only a sketch - “extended abstract- is expected) under an open-world scrutiny. 
On the contrary, journals expect articles to be definitive once published. Testing is 
done before by reviewers in a closed world. 

• Evaluation by hatch. Evaluation is done on a set of submissions at a time. Hence, 
implicitly the value of a paper is weighed against their peers in the same event. This 
method introduces the notion of ranking among submissions and bias in favor of fash¬ 
ions, gronps, and mainstream developments. 

• Changing board of editors. The notion of Program Committee (PC) is in my opinion one 
of the cornerstones of the system. Complemented by the Steering Committee ensnring 
long term stability, PC’s reflect and enforce the mobility of areas, topics, people and 
criteria. Today jonrnal editorial boards are rather closed and stable gronps, not withont 
inbreeding, limiting innovation. 

• Dissemination of ideas done by the producer. I think this is the kernel of the con¬ 
ference model, and a feature not sufficiently highlighted in the discussion on CvJ. 
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Conferences/workshops are (easily) organized by the interested researchers in a new 
held, who initiate a spiral of contribntions/evalnation/disseniination that nltimately 
will shape a new area if persisting in time. The key point is that there is no need of 
intermediaries, and the growth process parallels that of the commnnity. In this sense, it 
resembles the origins of scientihc societies that shaped cnrrent branches of knowledge. 
(There is a signihcant link between this featnre and current discussions on the future 
of publishing, open journals, etc.) 

• Physical presence, face-to-face meeting. This has been assumed to be the most dis¬ 
tinctive characteristic of the conference model. What is surprising and intriguing is 
that the community leading virtual communications and immaterial production is the 
one that seems more prone to physical meetings. Is this a contradiction? Let us re¬ 
mark that in our age of rapid communications, the other characteristics of conferences 
discussed do not need physical contact or presence. 

Being aware of the role and power of -and in favor of enforcing- face-to-face communi¬ 
cation and physical bonds, I rather disagree that it is the essence of what has rocketed 
the conference model. 

Final Comments. The conference model reflects fairly well our times. Like it or not, the 
hrst three characteristics discussed are typical of today’s social and economical rhythms, and 
£t very well modern time constraints in any area. Arguments stating that the conference 
“has fractured the discipline and skewered it towards short-term deadline driven research” 
point rightly to consequences, not to causes. In fact, these are general trends observable 
in all disciplines. Many of the aspects blamed to conferences are part of a new paradigm 
of science development (called “Mode 2” in the literature) characterized by the steering of 
research priorities, accountability of science and commercialization of research. This mode 
of development -their advocates state-, would have superseded the classical model (“Mode 
1”) characterized by the hegemony of theoretical or, at any rate, experimental science, by an 
internally-driven taxonomy of disciplines, and by the autonomy of scientists and their host 
institutions, the universities HD]. 

Dehnitively, when reading the CvJ debate, one cannot but bring to mind broader discus¬ 
sions on the transformation of the scientific method in our age. I think we should incorporate 
them into our discussion in order to fairly assess the contributions of conferences to current 
scientific developments. 
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